Identifying Abandoned, Moved and Removed Objects in Automated Surveillance Systems

This project is concerned with the classification of long term changes in surveillance scenes, known as static foreground. There has been little work related to the classification of static foreground in surveillance despite its potential role in automatic identification of abandoned bombs and theft. In this project, a thorough literature review of previous techniques was carried out. It was found that all previous static foreground classifiers were limited to classifying objects as either abandoned or removed. This leads to a large number of ambiguous cases whereby objects move, such as a bin falling over in the wind. I have implemented each of these classes of techniques and a comparison of their performance has been evaluated on a test set of 40 examples of abandoned and removed objects taken from established benchmarks. Classifying static foreground as an abandoned, removed or moved object presents new issues. Firstly, the static foreground observed no longer necessarily indicates the position of an object in the scene. Secondly, previous techniques relied on computing a metric from before and after the static foreground was detected. Comparing single metrics work in the constrained conditions of abandoned and removed objects, but break down for multi-class classification. In order to overcome these challenges a four stage system is presented. Leveraging the strengths of previous abandoned and removed classifiers, the system combines image inpainting, pixel level classifiers and image segmentation to create robust classifications. The first two stages overcome the issues associated with the static foreground of moved objects. The latter stages deal with the multi-class classification. The system was tested on 80 examples of abandoned, removed and moved objects and operated at 95% accuracy. Previous methods classified moved objects arbitrarily between abandoned and removed as the classification was outside their scope. The proposed system resolves this challenge and also matches the previous state of the art abandoned and removed object classifier in the absence of moved objects.